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BACKGROUND OF THE INVENTION 

Technical Field of the Invention 

The present invention relates generally to surveillance systems, and specifically to 
event detection using images captured by surveillance systems. 

Description of Related Art 

Surveillance systems commonly employ video cameras to monitor facilities. 
Historically, these cameras have transmitted analog video images of an area under 
surveillance to a security monitoring center for inspection and storage. In many facilities, 
analog video cameras are being replaced with digital cameras that detect and capture still 
images of events, such as the appearance of an intruder, a malfunction, or a fire within the 
area under surveillance. Digital cameras provide several advantages over analog video 
cameras. For example, digital cameras can be radio linked and battery powered to eliminate 
the need for the costly fixed infrastructure of video cables and power lines, making 
surveillance systems cheaper and easier to deploy. 

However, digital cameras have limited sensitivity, and in areas of low illumination or 
at night, a source of artificial illumination is required. In commercial applications, lights can 
be left on at night in order to illuminate the facilities that must be secured. However, in 
residential applications, lights that are left on at night can be disturbing to both the occupants 
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of a property and their neighbors. In addition, if a illumination source is attached to a 
battery-powered camera, constant illumination would reduce the life of the battery, 
necessitating frequent battery replacements. Therefore, what is needed is a surveillance 
system capable of operating under low illumination conditions. 

5 

SUMMARY OF THE INVENTION 

Embodiments of the present invention provide a surveillance system and method for 
detecting an event and capturing an image of the event under low illumination conditions. A 
camera is configured to capture a current image under low illumination conditions (e.g., 

1 0 between one centilux and one lux) and to compare the current image to a reference image 
previously captured under substantially similar low illumination conditions. The camera is 
further configured to determine, based on the results of the comparison, whether an event has 
occurred in the current image. The current image and reference image are captured using 
long exposure times (e.g., up to six seconds) in order to enable the camera to receive 

15 sufficient light in the low illumination conditions. 

In one embodiment, an illumination source is connected to the camera and activated 
upon the detection of an event to provide sufficient artificial illumination to capture one or 
more additional event images using a shorter exposure time (e.g., up to 1/60 of a second). 
The event image(s) can be stored internally within the camera or transmitted to an external 

20 security system for storage and/or analysis. 

In another embodiment, one or more thresholds are used to measure the change 
between the current and reference images required for an event to be detected. For example, 

245372v3 -? 



Patent Application 
Attorney Docket #10021083-1 
(47429-00 1 03/02 1 6 1 5 .0203) 

a difference value representing the difference between selected sensor values of the current 
image and selected sensor values of the stored reference are compared to a threshold. If the 
difference value exceeds the threshold, an event is detected. 

Advantageously, embodiments of the present invention enable a surveillance camera 
to operate under low illumination conditions without consuming excessive power. 
Furthermore, the invention provides embodiments with other features and advantages in 
addition to or in lieu of those discussed above. Many of these features and advantages are 
apparent from the description below with reference to the following drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The disclosed invention will be described with reference to the accompanying 
drawings, which show important sample embodiments of the invention and which are 
incorporated in the specification hereof by reference, wherein: 

FIG. 1 is a simplified overview of a surveillance system; 

FIG. 2 is a block diagram illustrating components of the surveillance system; 

FIG. 3 is a block diagram illustrating the operation of the surveillance system under 
low illumination conditions, in accordance with embodiments of the present invention; 

FIG. 4 is a block diagram illustrating exemplary logic for implementing an event 
detection module for use under low illumination conditions, in accordance with embodiments 
of the present invention; 

FIG. 5 is a flow chart illustrating an exemplary process for capturing an image of an 
event under low illumination conditions, in accordance with embodiments of the present 
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invention; and 

FIG. 6 is a flow chart illustrating an exemplary process for detecting an event under 
low illumination conditions, in accordance with embodiments of the present invention. 

5 DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 

The numerous innovative teachings of the present application will be described with 
particular reference to exemplary embodiments. However, it should be understood that these 
embodiments provide only a few examples of the many advantageous uses of the innovative 

10 teachings herein. In general, statements made in the specification do not necessarily delimit 
any of the various claimed inventions. Moreover, some statements may apply to some 
inventive features, but not to others. 

FIG. 1 illustrates a surveillance system 10 capable of operating under low 
illumination conditions, in accordance with embodiments of the present invention. The 

15 surveillance system 10 uses a camera 200, such as a digital camera, or other type of imaging 
sensor device to monitor the activity of targets in a scene 205 by capturing one or more 
images of the scene 205. The scene 205 that the surveillance system 10 monitors depends on 
a field-of-view 210 of the camera 200, which is determined, at least in part, by the type of 
lens that the camera 200 employs. For example, the lens can be a wide angle lens, capturing 

20 about a 60 degree field-of-view, or a regular lens, capturing about a 45 degree field-of-view. 

In embodiments of the invention, the camera 200 is capable of capturing an image 
under low illumination conditions, such as an image of a nighttime scene 205 with little or no 

245372v3 4: 



Patent Application 
Attorney Docket #10021083-1 
(47429-00 1 03/02 1615. 0203) 



artificial illumination. For example, the illumination level within a low illumination image 
can be between one centilux and one lux. Although low illumination images can be used to 
detect the occurrence of events, such as the appearance of an intruder, the resolution of 
images taken under low illumination conditions may be insufficient to adequately analyze the 
5 event because of motion blur. Therefore, a visible or infrared illumination source 150, such 
as a strobe light, is attached to the camera 200 or directly or wirelessly connected to the 
camera 200 (as shown) to provide temporary artificial illumination when an event occurs. 
For example, the illumination source 150 can be attached to a building, and activated by the 
camera 200 via a switch or wireless connection to a switch upon the detection of an event. 

10 The camera 200 is further capable of storing data 215 related to the captured image of 

the event for later use or retrieval. In other embodiments, the camera 200 is configured to 
transmit the data 215 to a security system 250 via a link 220 between the camera 200 and the 
security system 250. For example, the data 215 can include the whole event image or only a 
portion of the event image. The link 220 can include any transmission medium, such as 

15 coaxial cable, fiber optic link, twisted wire pair, air interface, satellite link or direct interface 
between the camera 200 and security system 250. The security system 250 can be physically 
located in a separate facility from the camera 200 or within the same facility as the camera 
200, depending upon the particular application. 

The security system 250 can be anything from a simple data storage device to a 

20 security monitoring center. For example, the monitoring center can include a computer 

capable of processing the data 215, displaying a picture in response to the data and producing 
a report related to the data. The computer can be a personal computer, server or other type of 
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programmable processing device. In other embodiments, the security system 250 can 
include an e-mail alert or other signaling method (e.g., paging) that is sent from the security 
system to a designated party. 

Referring now to FIG. 2; various components of the surveillance system 10 are 
5 illustrated. The surveillance system 10 includes an image sensor 20, such as a CMOS sensor 
chip or a CCD sensor chip within the camera 200 (shown in FIG. 1). The sensor 20 includes 
a two-dimensional array of pixels 25 arranged in rows and columns. In one embodiment, the 
image sensor 20 is a black and white sensor. However, in another embodiment, the image 
sensor 20 is a color sensor. 

10 The image sensor 20 is connected to provide raw sensor values 30 representing a 

current image to an image processing system 100. A storage medium 130 within the image 
processing system 100 stores a reference image 140 for comparison with the current image. 
In accordance with embodiments of the present invention, the reference image 140 includes 
sensor values representing a prior image taken under substantially similar illumination 

15 conditions as the current image. The storage medium 130 can be any type of memory device, 
such as a dynamic random access (DRAM) memory. 

A Central Processing Unit (CPU) 110 within the image processing system 100 
controls the receipt of the current sensor values 30 and compares the current sensor values 30 
to the stored reference sensor values 140 to determine whether an event has occurred. Upon 

20 the detection of an event, the CPU 110 further activates an illumination source 150 to provide 
sufficient artificial illumination to capture one or more additional high quality images of the 
event. The CPU 110 can be any microprocessor or microcontroller configured to load and/or 
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run software associated with the event detection module 120 and access the storage medium 
130. 

The operation of the surveillance system 10 under low illumination conditions is 
shown in FIG. 3. Under low illumination conditions (e.g., between one centilux and one lux), 
5 the sensor 20 captures a current image 35a using a long exposure time (e.g., up to 

approximately six seconds) and transmits sensor values representing the current image 35a to 
an event detection module 120 of the CPU 110. The exposure time is controlled by an 
exposure controller 320 that sets the photodiode integration time for each of the pixels in the 
sensor 20. With a long exposure time, the temporal resolution of moving objects within the 

10 captured image deteriorates due to image blurring resulting from the motion. However, the 
temporal resolution is sufficient to detect the occurrence of motion relative to a reference 
image 140 taken prior to the current image 35 a. 

To detect an event, the reference image 140 is retrieved from the storage medium 130 
and input to the event detection module 120 when the current image 35a is provided to the 

15 event detection module 120. In one embodiment, the stored reference image 140 includes 
sensor values representing a prior full digital image. In another embodiment, the reference 
image 140 is a set of statistics summarizing a prior digital image. When using a prior full 
image as a stored reference image 140, event detection may be based on either ratios or 
differences of sensor values with the current image 35a. For example, as described in 

20 Durucan and Ebrahimi, "Change Detection and Background Extraction by Linear Algebra," 
Proceedings of the IEEE, Vol. 89, No. 10, pp. 1368-1381 (2001), which is hereby 
incorporated by reference, a change between the current image and a reference image is 

245372v3 O 



Patent Application 
Attorney Docket #10021083-1 
(47429-001 03/02 1 6 1 5 .0203) 



detected based on a vector model of the current image compared to the vector model of the 
reference image. 

Once an event is detected, the event detection module 120 activates the illumination 
source 150, instructs the exposure controller 320 to reduce the exposure time and instructs the 
5 sensor 20 to take one or more subsequent event images 35b. As used herein, the term even 
image 35b includes one or more images. Under artificial illumination from the illumination 
source 150, the sensor 20 captures the event image 35b using a short exposure time (e.g., up 
to 1/60 of a second). The sensor values representing the event image 35b can be stored 
directly in the storage medium 130 or compressed and stored in the storage medium 130. 

10 The storage medium 130 used to store the event image 35b can be the same type of memory 
device used to store the reference image 140 or a different type of memory device, such as a 
compact flash or flash memory. In other embodiments, the event image 35b or other data 
related to the event image 35b is transmitted via a transmission interface 330 to an external 
security system for analysis and/or and storage. The transmission interface 330 provides an 

15 external interface to a wired or wireless connection to the external security system. 

Once the event image 35b are captured, the event detection module 120 deactivates 
the illumination source 150 and accesses the exposure controller 320 to return the exposure 
time to a long exposure time (e.g., up to six seconds). Activating the illumination source 150 
only when an event is detected minimizes power consumption and preserves battery life. 

20 The operation of the event detection module 120 in accordance with one embodiment 

of the present invention is shown in FIG. 4. The event detection module 120 is applied to the 
current sensor values 30 representing the current image 35a (shown in FIG. 3) taken under 
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low illumination conditions to determine whether an event has occurred in the current image. 
To detect an event, difference logic 400 retrieves reference sensor values 145 representing 
the reference image 140 (shown in FIG. 3) from the storage medium 130 and determines 
difference sensor values 405 that represent differences between the current sensor values 30 
5 and reference sensor values 145. For example, in one embodiment, the difference sensor 
values 405 are determined by subtracting the reference sensor values 145 from the current 
sensor values 30 on a pixel-by-pixel basis. Subtracting the reference sensor values 145 from 
the current sensor values 30 removes scene elements and dark current that are common to the 
reference and current images. In another embodiment, the difference sensor values 405 are 

10 determined by calculating ratios between the current sensor values 30 and the reference 

sensor values 145. It should further be understood that in the context of FIG. 4, and as used 
elsewhere below, the term "logic" refers to the hardware, software and/or firmware capable 
of performing the function of the logic. 

In one embodiment, the difference sensor values 405 are provided directly to 

15 comparison logic 440 to determine whether an event has occurred, as described in more detail 
below. In another embodiment, the difference sensor values 405 are first applied to spatial 
averaging logic 410 to calculate a difference value 415 determined by summing a certain 
number of the difference sensor values 405 and dividing by the number of summed difference 
sensor values 405. Averaging the difference sensor values 405 not only simplifies the 

20 analysis process, but also improves the discrimination between the signal component and the 
noise component within each of the sensor values, since the noise component typically varies 
from pixel to pixel. 
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The number of difference sensor values 405 used to determine the difference value 
415 depends on the type of sensor used, the exposure times of the sensor, the type of event 
being detected, and other factors. The particular difference sensor values 405 used to 
determine the difference value 415 depends on the areas of interest in the scene, the areas in 
5 the scene having known changes that may skew the results, the identification of defective 
pixels and other factors. 

The difference value 415 can be a single value or multiple values representing the 
average change within a spatial area of the current image. For example, the sensor values can 
be divided into blocks or regions of interest and a difference value can be computed for each 
10 block. When dividing the image into blocks, event detection can occur based on a change 
within a single block or changes over a number of blocks. 

The difference value 415 is applied to comparison logic 440 to determine whether the 
difference value 415 exceeds a threshold 430 provided by threshold logic 420. In one 
embodiment, the threshold 430 is a single threshold for comparison with a single difference 
15 value 415. In another embodiment, the threshold 430 includes multiple thresholds for 

comparison with multiple difference values 415. For example, if a separate difference value 
is computed for two or more blocks of pixels, an individual block threshold can be used to 
determine whether a particular block in the current image has changed significantly from the 
corresponding block in the reference image and a total block threshold can be used to 
20 determine whether the number of changed blocks indicates an event. 

In another example in which the difference sensor values 405 are applied directly to 
the comparison logic 440, the threshold 430 includes two thresholds, an individual pixel 
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threshold and a total pixel threshold. The individual pixel threshold is used to determine 
whether a particular pixel in the current image has changed significantly from the 
corresponding pixel in the reference image. The total pixel threshold is used to determine 
whether the total number of changed pixels indicates an event. The total pixel threshold can 
5 be applied to all pixel locations or only a portion of the pixel locations. For example, the 
total pixel threshold can indicate the number of pixels in an area or region of interest in the 
scene that need to change or the number of contiguous pixels in the largest contiguous 
collection of pixels that need to change to indicate an event. 

The threshold 430 for determining whether the change in the current image is 

10 significant enough to indicate an event can be preset for all images or computed based on the 
sensor values 30 of the current image. Thus, in some embodiments, the threshold 430 is set 
during the manufacturing process, by an operator of the surveillance system or using a table 
of values for the threshold 430 based on sensor values 30, etc. In other embodiments, the 
threshold 430 is fixed or pre-configured based on the image sensor being used. 

15 The comparison logic 440 outputs an event result 450 whose state indicates whether 

an event has occurred in the current image. If the event result 450 signifies that an event has 
been detected, the event detection module 120 activates the illumination source and one or 
more high quality images are collected using a short exposure time (e.g., up to 1/60 of a 
second). Otherwise, the current image is stored as the reference image and the event 

20 detection module 120 receives another long exposure image for comparison with the 
reference image. 

An exemplary process for capturing an image of an event under low illumination 
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conditions is shown in FIG. 5 . To detect an event, the camera captures a current image under 
low illumination conditions using a long exposure time (block 500). The current image is 
compared to a stored reference image taken under substantially similar illumination 
conditions prior to the current image (block 510). Thereafter, the stored reference image is 
5 updated with the current image for a subsequent comparison with a new image (block 520). 
Based on the comparison between the current image and the previously stored 
reference image, a determination is made whether an event has occurred (block 530). If an 
event is detected in the current image (block 530), an illumination source is activated (block 
540) to provide sufficient artificial illumination to capture one or more high quality images of 
10 the event using a short exposure time (block 550). Execution then returns to block 500. If 
the result of the comparison indicates an event has not occurred in the current image (block 
530), a new long exposure image is captured (block 500) and compared with the stored 
reference image (block 5 1 0), which now corresponds to the immediately preceding current 
image. 

15 An exemplary process for detecting an event within a current image (block 530) is 

shown in FIG. 6. Upon receiving the sensor values for the current image (block 600), 
reference sensor values corresponding to a previous image are retrieved (block 610) for 
comparison with the current sensor values. A difference value between the current sensor 
values and the reference sensor values is calculated (block 620) for use in determining 

20 whether a change has occurred in the current image compared with the previous image. 

If the difference value exceeds a threshold (block 630), an event is detected in the 
current image (block 640). Upon the detection of an event, an event notification is 
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transmitted to indicate that an event has occurred (block 650). If the difference value does 
not exceed the threshold (block 630), no event is detected (block 660). 

The innovative concepts described in the present application can be modified and 
varied over a wide range of applications. Accordingly, the scope of patented subject matter 
5 should not be limited to any of the specific exemplary teachings discussed, but is instead 
defined by the following claims. 
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